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This appeal is responsive to the Final Rejection of March 
16, 2001 (Paper No. 7) , and the Office Advisory Action of May 
23, 2001 (Paper No. 9) . 



The Commissioner is hereby authorized to charge any fees 
in connection with this appeal or with other matters before 
the PTO to the undersigned's Deposit Account No. 19-0077. 



08/22/2001 HVU0MG1 00000004 190077 09134981 
01 FC:220 155.00 CH 



Docket No. H-409 



1 



REAL PARTY IN INTEREST 



The real party in interest hereto is Appellant's 
assignee. The interest in the invention was assigned by the 
inventor to Soft Sight, Inc. at the time of filing the 
application, and was recorded on Reel No. 9464, Frame No. 



RELATED APPEALS AND INTERFERENCES 




This appeal is the first appeal before the Office. 



STATUS OF THE CLAIMS 

Claims 1 through 9, 11 through 14, 16 through 18, and 2 0 
now stand FINALLY REJECTED. Claims 15 and 19 are indicated 
allowable, but presently dependent upon rejected claims. The 
FINAL REJECTION (Paper No. 7), and the Advisory Action of 
(Paper No. 9) are in error as to the status of the claims. 
Claim 10 was canceled in the amendment filed January 4, 2001. 
With all due respect, the Office has not acknowledged this 
cancellation. 
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The rejection of claims 1 through 9, 11 through 14, 16 
through 18, and 2 0 is hereby appealed. 

STATUS OF THE AMENDMENTS 

The application was filed on August 17, 1998. An Office 
Action was mailed on October 3, 2000 (Paper No. 5) . An 
amendment was filed on January 2, 2001 (Paper No. 6) in 
response. The FINAL REJECTION was mailed on March 16, 2001 
(Paper No. 7) . An Office Interview was held on May 15, 2001 
among Appellants attorney, Mark Levy, the inventor, 
Dr. David Goldman, and Examiner Tim Vo. An amendment after 
FINAL REJECTION was HAND CARRIED to Examiner Tim Vo, and 
presented to him at the Office Interview. 

An Office Advisory Action, apparently erroneously citing 
a non-analogous patent with which Applicant was heretofore 
unfamiliar, was mailed on May 23, 2001, but was not received 
by Appellant 1 s attorneys. Examiner Tim Vo was consulted, and 
kindly faxed a copy of this action to Robert Salzman, one of 
the attorneys of record on or about August 6, 2001. 
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SUMMARY OF THE INVENTION 



The invention comprises a method and system for 
computerized embroidery of scanned color images. Generally, 
the present invention supplies a file containing embroidery 
data (from a scanned color image) to an embroidery- generating 
program. In turn, the embroidery-generating program executes 
a series of sub-routines with respect to the embroidery data 
file, and produces an embroidery output file, which contains 
data that is fed to a sewing device for stitching the 
embroidery design. 

The embroidery-generating program of this invention has 
many complex features, owing to the fact that a scanned color 
image that is to be converted into an embroidery pattern must 
account for many subtle distinctions: color shading, line 
thickness, and other subtle characteristics that the eve can 
perceive, but which are often irreproducible by the common 
computer program, particularly those processing only black and 
white data . 

The sub-routines, which are the heart of the current 
invention, contain a segmentation mechanism, a chain-coding 
mechanism, a distance transform computation mechanism, a line- 
fitting mechanism, a stitch angle determination mechanism, a 
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fragment generating mechanism, an embroidery output mechanism, 
a fitting mechanism, a labeling mechanism, a merging 
mechanism, a coding mechanism, a column smoothing mechanism, 
and a path generating mechanism. 

When a user first scans a color embroidery pattern, the 
software program within the scanner creates a representation 
of the pattern, referred to as embroidery data. The 
embroidery data is broken up into small objects called pixels. 
Each pixel in the embroidery data has a bitmap associated with 
the color of the pattern. Each unique color in the scanned 
pattern has its own unique bitmap. Before the embroidery 
generating mechanism executes the embroidery data, embroidery 
data is in its raw form, and the bitmapped data stored within 
the file has not yet been preprocessed. For example, some 
pixels may have incorrect bitmap assignments due to scanner 
noise or other subtle variations in color, such variations 
being accounted for by the human eye but their distinction 
becoming non-trivial when dealt with procedurally by a 
computer program. 

Once embroidery data has been scanned, the embroidery 
generating mechanism executes a segmentation routine that 
classifies each pixel within embroidery data into a distinct 
object. Each object represents a general color in the data. 
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Any pixels that were not categorized into a particular object 
during the first segmentation may be considered noise, 
generated during scanning, when the scanner could not properly 
assign a bitmap for a particular pixel, as would be likely 
achieved by a human counterpart. 

Each pixel has a series of eight neighboring pixels. It 
is likely that at least one of these neighboring pixels has 
been assigned to a particular object. The segmentation sub- 
routine assigns an uncategorized pixel to an object, where a 
neighboring pixel has been assigned. It should be understood 
that one or more of the eight neighboring pixels might belong 
to different objects. The segmentation mechanism uses" 
mathematical methods to determine to which object an 
uncategorized pixel should be assigned. This is where black 
and white, versus color processing, is most different, because 
not limiting a system to only two values (black and white) 
introduces substantial complexity to the automated embroidery 
problem as it relates to image segmentation. 

Once the segmentation sub-routine segments embroidery 
data into objects, the chain-coding sub-routine performs 
further image processing ' on each individual object within the 
embroidery data. The chain-coding sub-routine codes each 
object's edge contours, and then uses the contours to compute 
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the skeleton and integer distance transform, DT, for each 
object . 

The system classifies each object into one of two 
categories. Either the object is a thick, predominantly 
singular object, or a thin, predominantly regular object. 
Whether the object is classified as a thick or a thin object 
determines the next series of processing steps needed to 
generate output embroidery data for each object. 

The system will then execute different processing 
routines depending on the shape characterization. Several 
processing routines embody the notion of labelling or 
identifying unique points on the skeleton and edge contours of 
a shape. Furthermore, these separate skeletal and edge 
contour points are interrelated: a particular point on an edge 
contour may be deemed to belong to a particular point on a 
related skeletal contour, like a jigsaw puzzle piece belongs 
to a particular location within the puzzle scene. The use of 
interrelating skeletal points with edge contour points forms 
one of the primary foundations of the invention. It is these 
interrelations that allow singular and regular regions to be 
identified within shapes and that allow subsequent embroidery 
to be generated. 
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ISSUES 



All of the claims of the application, but for claims 15 
and 19, have been rejected as being anticipated by FUTAMURA 
(United States Patent No. 5,740,056) under 35 U.S.C. §102 (e) . 

GROUPING OF THE CLAIMS 

The claims should not be grouped together, because many 
of the dependent claims contain processing features not 
contained in the independent claims. Some of these processing 
features are unique to color stitching, and are respectfully 
not shown or suggested by the black and white embroidery 
program illustrated by the cited art of record. Furthermore, 
with all due respect, the Office has failed to rebut 
Applicant's arguments in many respects regarding both 
independent and dependent claim features and differences, 
either appearing to ignore, gloss-over, or misinterpret 
salient arguments and claim language presented by Appellant. 
Fairness and completeness should dictate a careful reading of 
all of the claimed features represented by Applicant's 
independent and dependent claims. 

Appellant ! s attorneys respectfully suggest a grouping of 
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claims 1 through 11, excluding cancelled claim 10; claims 12 
through 14 and 16 through 18; and claim 20. Independent claim 
1 calls for interrelating skeletal and edge contours, which 
clearly relate to the processing of an image as taught by 
Appellant. Independent claim 12 recites "classifying and 
line-fitting" with respect to both edge and skeletal contours 
for thin objects or thick objects. Independent claim 20 
recites locating and interpreting a set of regular and 
singular regions, and computing an optimum sew order, as 
befits embroidery. These independent claims should not be 
lumped together, because they deal with different aspects of 
Appellant ' s method . 



ARGUMENT 



The Honorable Board is respectfully requested to consider 
that in order for an application to be considered anticipated 
by the cited art under 35 U.S.C. §102 (e) , there must be 
present a complete teaching of all of the salient features of 
Appellant's claimed process. Appellant, through his 
attorneys, contends that the FUTAMURA reference does not 
present so fine and complete a teaching in accordance with 
section 102 of title 35, that "anticipation" is proven in 
accordance with the statute. 
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Appellant has made the following arguments, to which the 
FINAL REJECTION was respectfully in error. 

Appellant has generated a program that is bifurcated with 
respect to processing black and white images, and color 
images. Appellant noted from the beginning that FUTAMURA 
processed only black and white embroidery. The Office 
maintained that black and white where colors indicative of 
meeting the claim language of claims 1, 12, and 20. This is 
respectfully a specious argument built upon semantics. 
"White" does represent a color, but "black" is defined as a 
shade. However, Appellant does not desire to rely on 
semantics, but rather wishes to suggest to the Honorable Board 
that the claims be interpreted with respect to the teachings 
in the specification, in which different processing is shown 
for embroidery in color (i.e., colors of the spectrum) vis-a- 
vis black and white. Please see In Re Buehler , (CCPA 1975) 
515 F2d 1134, 185 USPQ 781. It is not proper for the Office 
to suggest that FUTAMURA meets the claim language of "color" 
by showing black and white. Appellant 1 s specification must be 
used to interpret the claims, and Appellants specification 
defines color as "colors of the spectrum" . Also, please see 
In re Boe et al , (CCPA 1974) 505 F2d 1217, 184 USPQ 38. 
"Limitations in claims distinguishing over the prior art 
cannot be ignored." In processing color, different problems 
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present themselves to Appellant that are not addressed by 
FUTAMURA. These problems include factors relating to image 
segmentation, a concept not mentioned in FUTAMURA. 

Moreover, FUTAMURA is silent with respect to shape 
characteristics, such as: intersecting of multiple pattern 
lines (i.e., singular regions), irregular, and non-uniform 
boundaries, fill stitching, etc. Independent claim 1 calls 
for creating and interrelating skeletal and edge contours, 
which clearly relate to processing a color, scanned image as 
taught by Appellant. The Office neglected to address this 
issue entirely , notwithstanding the fact that it forms a crux 
of the invention and is clearly not anticipated by FUTAMURA. 
The claims were amended (see claim 1(c)) and appropriately 
argued, but were never acknowledged by the Office. 

Independent claim 12 recites "classifying and line- 
fitting" with respect to both edge and skeletal contours for 
thin or thick objects. Again, FUTAMURA does not anticipate 
such a combination of classifying and line-fitting with the 
use of edge and skeletal contours. 

Independent claim 20 recites locating and interpreting a 
set of regular and singular regions, and computing an optimum 
sew order, as befits embroidery. FUTAMURA neither identifies 
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nor processes regular and singular regions. 

Comparing the two flow diagrams of each invention, it 
clearly becomes evident that Appellants method is much more 
complex than FUTAMURA. How FUTAMURA can be equated as 
anticipatory of Appellant's method, respectfully defies logic. 

Claim 2 teaches the sub-routine process of segmenting, 
which characterizes each pixel of an image data file as an 
object. In color, scanned data requires image segmentation. 
Appellant had submitted Exhibits A and B to illustrate the 
point that no segmentation was performed by FUTAMURA. In 
FUTAMURA, such data was utilized directly, potentially 
resulting in erroneous outcomes, as indicated in Exhibit A. 
Data in FUTAMURA was processed irrespective of pixels 
representing noise, half tones, and other anomalies. In claim 
2, paragraph (c) , Applicant recites means for locating and 
interpreting a set of regular and singular regions for 
embroidery point generation, which is not shown in column 5, 
lines 19 through 24 of FUTAMURA , as the Office contends. The 
description in column 5, lines 19 through 24 of FUTAMURA 
refers to scanning the image and storing it in RAM. The 
Office has respectfully twisted Appellant f s wording in the 
claim, so that FUTAMURA can be read into Appellant 1 s 
invention. In claim 2, paragraph (d) , neither is the "path 
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generation" nor the "optimum sew order" for at least one 
extracted column, shown in column 5, lines 51 through 67, as 
the Office contends. 

Most importantly, there is no path planning in FUTAMURA. 
This is a very important processing step for Appellant. 

Appellant can go on distinguishing the differences, but 
it would be gilding the lily. It should be obvious at this 
point that FUTAMURA cannot meet the burden of "anticipation, " 
in accordance with 35 U.S.C. §102. 

Considering claims 3 and 18, FUTAMURA does not teach 
line- fitting in combination with both edge and skeletal 
contours, in the sense taught by Appellant. 

Considering claims 4, 5, 13 and 14, please observe that 
FUTAMURA does not teach stitch angle determination, which is 
part of fragment ordering, that is important in generating 
fill stitch objects. In fact, FUTAMURA does not generate 
objects of this type by any method. 

Considering claim 6, FUTAMURA does not teach labelling a 
plurality of points individually belonging to a shape's 
skeleton and edge contours . 
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Considering claim 7, FUTAMURA does not teach "merging a 
series of points on the skeleton contour." 

Considering claim 8, FUTAMURA does teach "coding means 
for evaluating a plurality of singular regions." 

Considering claim 9, FUTAMURA does not teach "smoothing 
means for evaluating a sequence of stroke normals." 

Claim 11 recites selecting a uniform color selected from 
a group of colors in the continuous spectrum. FUTAMURA, who 
processes only black and white, is definitely non- 
anticipatory. 

In Paper No. 7, page 2, the Office states that the 
recitation: "fill stitch, and handling cases where multiple 
pattern lines intersect (i.e., singular regions within the 
pattern)," was not found in Appellant's claims, as a means of 
blunting Appellant's claimed differences over the cited 
FUTAMURA. Claims 8 and 20, while not reciting the exact 
wording, do speak of interpreting and generating an output 
file dependent on a set of regular and singular regions. What 
the Office neglects is that FUTAMURA does not teach a method 
that even considers the claimed coding of regular and singular 
regions, and therefore certainly cannot be considered 
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anticipatory. 

Finally, the reference by the Office pursuant to the 
Advisory Action of May 23, 2001 (U.S. Patent No. 5,444,640) 
appears to be an error on the part of the Office, as it 
relates to a numerical control method correcting for thermal 
displacement, which obviously has nothing to do with 
embroidery. 

In summary, the Office suggests that many of the claimed 
differences suggested by Appellant are not in the claims, and 
cannot be read into the claims by way of specification 
teachings. But Appellant's claims do recite differences 
sufficient for their being allowable. Detail, when required, 
is generally left to the specification. Appellant argues that 
having made recitation of a defining difference, the 
specification can be used to fill in any interpretive meaning. 

CONCLUSION 

The Honorable Board is respectfully requested to reverse 
the rejection of claims 1 through 9, 11 through 14, 16 through 
18 and 20, and allow the subject application to issue as a 
patent . 
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Respectf ully submitted, 



I hereby certify that this correspondence is being 
deposited with the United States Postal Service as first 
class mail in an envejopp addressed to: 
Commissioner oTratents and Tradernafks 
Washington, D.0#>O231 , 




Mark Levy, Reg, 
Attorney 




29, 188 



Mark Levy 
Registration No, 
Attorney for Appellant 
SALZMAN Sc LEVY 
Press Building - Suite 902 
19 Chenango Street 
Binghamton, New York 13 901 



Phone: (607) 722-6600 
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APPENDIX 



1. A system for automatically producing an embroidery 
design, the system comprising: 

a) means for inputting an embroidery pattern into an 
image data file, the image data file comprising a 
plurality of pixels, each pixel comprising a 
bitmap representing a color; 

b) processing means operatively connected to said 
inputting means for creating skeletal and edge 
contour data and storing said image data file; and 

c) an embroidery data generating mechanism 
operatively connected to said processing means for 
labeling and interrelating said skeletal and edge 
contour data and generating a complex embroidery 
pattern directly from a scanned, color image. 

2. The system of claim 1, wherein said embroidery data 
generating mechanism comprises: 

i) segmenting means for characterizing each 
pixel within the image data file into an 
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ii) means for classifying each of said 
objects as a thin object or a thick 
object ; 



iii) means for locating and interpreting a set of 
regular and singular regions for embroidery 
data point generation. 



iv) path generation means for computing an 
optimum sew order for at least one 
extracted column; and 



v) embroidery output means for generating 
an embroidery output file. 



3. The system of claim 1, further comprising line- 
fitting means for line- fitting each object, wherein an 
object comprises an outer contour, a predetermined number of 
inner contours, and a skeleton contour, said line-fitting 
means comprising a gallus-neurath triangular filter. 



4. The system of claim 3, further comprising stitch 
angle determination means for determining a stitch angle 
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that produces a minimal plurality of fragments. 

5. The system of claim 4, further comprising 
generate path means for determining an optimal order for the 
plurality of fragments to be sewn. 

6. The system of claim 1, further comprising labelling 
means for labelling a plurality of points on the skeleton 
and edge contours . 

7. The system of claim 6, further comprising merging 
means for merging a series of points from the plurality of 
points on the skeleton contour. 

8. The system of claim 7, further comprising coding 
means for evaluating a plurality of singular regions. 

9. The system of claim 8, further comprising smoothing 
means for evaluating a sequence of stroke normals. 

11. The system of claim 1, wherein said object 
comprises a plurality of connected or contiguous pixels 
having a uniform color. 
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12. A method for automatically producing an embroidery- 
design, the method comprising the steps of: 

a) inputting an embroidery pattern into an image data 
file, the image data file comprising a plurality 
of pixels, each pixel comprising a bitmap 
representing a color; 

b) classifying and line-fitting each object in said 
bitmap as a thin object or a thick object, each of 
said objects comprising an outer contour, any 
number of inner contours, and a skeleton contour; 

c) computing an optimum sew order; and 

d) generating an image output file. 

13. The method of claim 12 further comprising the step 
of generating the plurality of fragments. 
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14. The method of claim 13 further comprising the step 
of determining an optimal order for the plurality of 
fragments to be sewn. 

16. The method of claim 15 further comprising the step 
of extracting at least one column. 

17. The method of claim 16 wherein said step (b) of 
classifying each pixel within the image data file comprises 
the step of associating each connected pixel having a 
similar color with a unique object identity. 

18. The method of claim 17 wherein said step (b) of 
classifying additionally comprises the step of traversing a 
plurality of chain codes associated with one of the group of 
skeleton contour, inner contour (s) , and outer contour. 

20. A method for automatically producing an embroidery 
design, the method comprising the steps of: 



a) inputting an embroidery pattern into an image 



data file, the image data file comprising a 



plurality of pixels, each pixel comprising a 



bitmap representing a color; 
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b) locating a set of regular and singular regions 
disposed in said image data file; 

c) interpreting said set of regular and singular 
regions ; 

d) computing an optimum sew order; and 

e) generating an image output file dependent on 
said interpreted set of regular and singular 
regions . 



END OF APPENDIX 
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